Proceedings  of  DETC’01: 

ASME  2001  Design  Engineering  Technical  Conferences  and  Computers  and  Information 

in  Engineering  Conference 
Pittsburgh,  Pennsylvania,  September  9-12,  2001 


DETC2001/CIE21267 

DESIGN  SYNTHESIS  IN  A  VIRTUAL  ENVIRONMENT 


Douglas  Maxwell** 
Louisiana  Tech  University 
PO  Box  10348 
Ruston,  LA  71272 


Rob  King 

Naval  Research  Laboratory 
4555  Overlook  Ave.,  SW 
Washington,  DC  20375 


Alley  Butler  * 

Louisiana  Tech  University 
PO  Box  10348 
Ruston,  LA  71272 

Keywords:  virtual  reality,  design  synthesis,  immersive  environments,  CAD 


ABSTRACT 

Traditional  computer-aided  design/engineering 
(CAD/CAE)  is  performed  in  a  two-dimensional  (2D) 
environment.  Through  the  CAD  system  demonstration 
described  in  this  paper,  an  alternative  to  the  existing  2D 
technology  for  design  of  products  is  described.  This 
alternative  in  which  design  synthesis  is  accomplished  in  a 
stereoscopic  or  immersive  environment  is  called  the  Design 
Synthesis  Virtual  Environment  or  DSVE.  Stereoscopic  vision 
permits  depth  perception  and  provides  perception  of  “real 
world”  3D  existence.  This  DSVE  technology,  as  defined  in 
this  paper,  provides  a  virtual  or  immersive  environment, 
computer-generated,  but  seemingly  real. 

The  authors  argue  that  significant  advantages  are  possible 
when  an  immersive  environment  such  as  the  DSVE  is  used  for 
design  synthesis.  In  the  Design  Synthesis  Virtual 
Environment  a  gestural  interface  is  used,  and  it  is  further 
argued  that  designers  can  perform  design  activities  intuitively 
by  using  a  more  natural  gestural  interface  within  the  design 
environment,  allowing  more  rapid,  efficient  actions  to  create 
and/or  modify  product  geometry. 

As  evidence,  this  paper  reports  on  a  demonstration 
conducted  in  a  Design  Synthesis  Virtual  Environment. 
Operation  of  the  DSVE  is  explained,  and  two  example 
problems  are  worked.  Conclusions  are  drawn  about  the 


effectiveness  of  the  DSVE  and  the  potential  for  further 
development. 

INTRODUCTION 

Traditional  computer-aided  design/engineering 
(CAD/CAE)  is  performed  in  a  two-dimensional  (2D) 
environment  using  systems  ranging  from  “low  end”  personal 
computers  to  relatively  high-powered  computer  workstations. 
The  work  accomplished  in  this  environment  can  be  rendered  in 
three-dimensions  (3D)  using  perspective  view  algorithms  and 
appropriate  illumination  models.  However,  the  display  still 
exists  in  2D,  because  the  display  device  (the  CRT  screen) 
typically  does  not  permit  immersive  viewing.  Through  the 
CAD  system  demonstration  described  in  this  paper,  an 
alternative  to  the  existing  2D  technology  for  design  of  products 
is  described  in  which  design  synthesis  is  accomplished  in  a 
fully  immersive  virtual  environment  called  the  Design 
Synthesis  Virtual  Environment  or  DSVE. 

Immersion  is  the  sense  of  presence  that  is  created  when 
the  reality  of  the  virtual  environment  approaches  that  of  the 
real  world.  Essential  elements  for  full  immersion,  as  opposed 
to  so-called  desktop  immersion,  are  stereoscopic  perspective 
vision,  a  display  that  permits  the  user  to  look  in  any  direction, 
passive  head  and  hand  tracking,  and  graphic  computing  power 
sufficient  to  achieve  an  adequate  update  rate.  In  such  a 
system,  the  user  perceives  “real  world”  3D  existence.  The 
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DSVE  technology  described  in  this  paper  not  only  provides  an 
immersive  environment  for  visualization  but  for  interaction  as 
well. 

To  date,  immersive  environments  have  been  used  for 
viewing  and  analysis  of  completed  designs.  Immersive 
environments  are  more  widely  used  for  training,  and  for 
simulation-based  design  in  which  geometry  has  been 
synthesized  by  traditional  means.  However,  experience  with 
synthesis  in  an  immersive,  tracked  stereoscopic  environment 
has  been  significantly  limited.  The  research  reported  upon  in 
this  paper  responds  directly  to  the  lack  of  experience  by  direct 
experimentation  with  synthesis  of  product  geometry  in  the 
DSVE. 

Immersive  virtual  reality  design  systems  hold  the  promise 
of  significant  savings  in  costs  and  time-to-market  for  new  part 
designs.  By  coupling  design  and  prototype  activities, 
practitioners  diminish  designer  effort  and  time  on  task  while 
holding  the  promise  of  better  decision-making  and  improved 
design  solutions. 

The  creation  of  a  virtual  reality  based  application  such  as 
DSVE  exposes  engineers  to  an  entirely  new  paradigm  for 
product  design.  Rather  than  using  abstract  algorithms  for 
solid  modeling  of  products,  a  designer  performs  design 
activities  more  intuitively  in  a  virtual  environment.  It  is 
argued  that  the  use  of  gestures  and  hand  movements  provides  a 
more  natural  interface  with  the  design  environment,  allowing  a 
more  rapid,  efficient  set  of  actions  to  create  and/or  modify 
product  geometry. 

Accordingly,  the  goal  of  the  research  reported  upon  in  this 
paper  is  to  demonstrate  application  of  the  DSVE  to  accomplish 
design  synthesis  of  complex  mechanical  systems  such  as  ships 
and  aircraft.  The  fundamental  premise  of  this  research 
involves  the  use  of  tracked,  stereoscopic  perspective  vision  for 
viewing  and  interacting  with  an  engineering  design 
environment,  the  DSVE.  Additionally,  gestural  interfaces  are 
used  to  provide  an  improved  means  of  interaction.  The 
expectation  is  that  the  DSVE  which  uses  gestural  interfaces 
offers  the  designer  significantly  improved  opportunities  for 
efficient  interaction  with  the  product  being  designed.  The 
DSVE  also  provides  stereoscopic  perspective  vision,  an 
advantage  in  design  tasks  by  engineers. 

In  describing  this  research  project,  this  paper  begins  with 
a  review  of  the  literature  to  provide  context.  The  hardware 
and  software  used  to  operate  the  DSVE  are  discussed  next.  An 
overview  of  DSVE  operation  is  provided,  and  the  use  of  the 
DSVE  in  two  demonstrations  is  then  discussed,  followed  by 
conclusions. 


LITERATURE  REVIEW 

The  field  of  virtual  systems  is  a  relatively  new  field  of 
study,  since  the  hardware  and  software  has  only  been  available 
during  the  last  ten  years.  For  this  reason,  the  literature  is 


limited,  but  there  are  several  important  areas  of  research  that 
have  emerged. 

Waurzyniak  [1]  outlines  important  advantages  to  virtual 
modeling  and  calls  for  better  performing,  less  expensive  virtual 
prototyping  systems.  Important  advantages  outlined  include 
the  elimination  of  physical  prototypes,  which  can  save 
automakers  and  aircraft  suppliers  millions  of  dollars  per  year. 
Another  advantage  is  the  potential  to  lessen  the  time  needed  to 
bring  new  products  to  market. 

Gomes  de  Sa  [2]  provides  some  important  definitions  of 
virtual  prototyping.  Virtual  prototyping  is  the  application  of 
virtual  reality  for  prototyping.  Another  aspect  of  virtual  design 
and  prototyping  is  the  promise  of  economic  savings.  Digital 
mock-ups  are  a  realistic  computer  simulation  of  a  product  with 
the  capability  of  all  required  functionality  from 
design/engineering,  manufacturing,  product  service 
environment,  maintenance,  and  product  recycling.  Digital 
mock-ups  are  used  as  a  platform  for  product  and  process 
development,  for  communication,  and  making  decisions  from  a 
first  conceptual  layout.  Werkhoven  [3],  Boudreaux  [4],  and 
others  have  studied  virtual  environments  for  designing  ships. 
Werkhoven  lists  areas  in  which  the  use  of  virtual  environments 
has  proven  to  be  worth  the  cost. 

Balaguer  describes  a  pilot  project  called  Virtual 
Environment  Navigation  in  the  Underground  Sites  (VENUS) 
for  virtual  prototyping.  VENUS  is  composed  of  four 
applications.  They  are  virtual  prototyping,  networked  design 
integration,  territory  impact  study,  and  assembly  planning  and 
control.  Balaguer  [5]  points  out  that  binocular  perception 
allows  for  better  understanding  of  the  complex  spatial 
relationships  between  parts  of  a  prototype.  He  comments  on 
the  current  capabilities  of  today’s  CAD  tools  as  being  too 
limited  to  allow  interactive  navigation.  In  addition  to  limited 
navigation  capabilities,  eventual  design  errors  are  difficult  to 
isolate. 

Jayaram  et  al.  [6]  describes  a  virtual  reality  based 
engineering  application  that  allows  engineers  to  plan,  evaluate, 
and  verify  the  assembly  of  mechanical  systems  called  the 
Virtual  Assembly  Design  Environment  (VADE).  VADE 
represents  collaboration  between  the  University  of  Washington 
and  the  National  Institute  of  Standards  and  Technology 
(NIST).  The  virtual  environment  within  VADE  presents  the 
user  with  an  assembly  scene.  The  tools  used  in  the  assembly 
process,  such  as  screwdriver  and  wrench,  are  located  where 
they  would  be  in  a  real  assembly  facility.  These  locations  are 
user  defined.  VADE  uses  a  virtual  hand  to  facilitate  assembly 
and  interaction  with  the  system  is  accomplished  with  a 
CyberGlove. 

Arangarasan  and  Gadh  [7]  describe  a  system  called 
Detailed  Virtual  Design  System  (DVDS)  under  development 
for  detailed  shape  modeling  in  a  multi-modal,  multi- sensory 
virtual  environment.  The  DVDS  provides  an  intuitive  and 
natural  way  for  mechanical  design  using  hand  motion  and 
voice  commands.  Work  with  DVDS  has  focused  on  the  use  of 
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data  gloves  and  voice  commands,  and  the  research  effort  of 
Aranigarasan  and  Gadh  has  placed  emphasis  on  collaboration 
of  design  teams  over  a  network  where  designers  are  in  separate 
locations. 

In  earlier  work  conducted  at  the  Navy  Research 
Laboratory,  Summers  et  al.  [8]  discussed  a  feature  based 
virtual  system.  The  feature-based  system  is  a  direct 
predecessor  of  the  work  reported  in  this  paper,  and  many  of  the 
research  goals  expressed  in  the  earlier  paper  have  been 
realized  through  the  DSVE  system. 

Although  the  VADE,  VENUS,  and  DVDS  projects 
represent  important  new  applications  of  virtual  environments, 
it  is  apparent  that  experimentation  in  the  use  of  virtual  systems 
for  engineering  design  purposes  has  been  significantly  limited. 
This  brief  review  has  summarized  work  in  virtual  prototyping, 
applications  of  virtual  environments  to  ship  design,  and  early 
systems  for  design,  development,  and  analysis.  As  one  such 
system,  the  Design  Synthesis  Virtual  Environment  research 
effort  reported  here,  adds  to  the  literature  on  the  use  of  virtual 
environments  for  engineering  design. 


HARDWARE  AND  SOFTWARE 

The  demands  placed  on  the  computer  and  graphics 
subsystems  for  a  virtual  environment  are  substantial  due  to  the 
higher  numbers  of  polygons  to  render  in  real-time  and  the 
higher  frame  rates  involved  to  support  stereoscopic  vision.  A 
magnetic  tracking  system  is  needed  to  facilitate  gestural  input, 
object  picking  in  three  dimensions,  and  correct  user- 
perspective  viewing.  These  processes  add  additional  tasks  to 
the  computer  hardware,  requiring  significant  computational 
power. 

The  high  fidelity  graphics  are  generated  by  an  SGI  Onyx  2 
with  two  graphic  pipes,  and  four  raster  managers.  The 
software  employed  includes  the  Irix  Version  6.5  Operating 
system,  SGI  Performer  Version  2.2  Graphics  library,  and 
OpenGL.  Additionally,  the  Virtual  Reality  Peripheral  Network 
software,  described  below,  is  used  to  control  the  tracking  and 
interface  device  systems.  The  display  device  is  the  GROTTO, 
installed  at  the  Naval  Research  Laboratory  (NRL). 

The  GROTTO  (Graphical  Room  for  Orientation, 
Training,  and  Tactical  Observation)  is  a  Surround-Screen 
Visualization  Environment  (SSVR™)  system  developed  by 
Mechdyne  Corporation.  The  GROTTO  uses  folded  optics  to 
project  images  from  CRT  projectors  onto  translucent  wall  and 
white  floor  surfaces.  The  images  are  produced  on  left,  front, 
and  right  walls  by  back  projection  and  on  the  floor  by  direct 
projection.  Crystal  Eye  shutter  glasses  are  used  to  produce  a 
stereoscopic  view.  The  user’s  position  within  the  GROTTO  is 
tracked  by  an  Ascension  Flock  of  Birds  (FOB)  magnetic 
tracking  system.  A  sensor  is  attached  to  the  user’s  eyewear 
and  a  sensor  is  attached  the  flightstick. 

The  flightstick,  shown  as  Figure  1,  is  the  primary  means 
of  interacting  with  the  immersive  virtual  reality  system.  It 


consists  of  a  PC  computer  joystick  modified  by  the  addition  of 
the  magnetic  tracker  as  well  as  re-wiring  the  four  buttons  to 
interface  via  an  Immersion  Technologies  I-box.  Logically,  the 
flightstick  is  a  six  degree-of-freedom  tracked  device  with  four 
buttons. 

The  I-box  allows  connection  of  up  to  eight  input  devices, 
four  digital  and  four  analog,  to  any  computer  with  a  serial  port. 
The  host  computer  accesses  the  I-box  through  standard  serial 
communications. 

The  tracking  system  and  I-box  interfaces  employ  the 
Virtual  Reality  Peripheral  Network  (VRPN)  client-server 
software,  provided  by  the  University  of  North  Carolina,  Chapel 
Hill  with  funding  from  the  National  Science  Foundation. 

A  user  operates  the  CAD  system  while  standing  inside  the 
GROTTO.  Although  the  primary  interaction  uses  the 
flightstick,  the  keyboard  and  mouse  are  still  needed  to 
communicate  with  the  computer  for  some  activities.  Leaving 
the  GROTTO  to  access  the  keyboard  is  an  inefficient  and 
unproductive  action.  A  lightweight,  portable  stand  called  the 
pulpit  is  constructed  of  PVC  pipe  and  plywood  to  support  the 
keyboard  and  mouse  inside  the  GROTTO. 


OVERVIEW  OF  DSVE  OPERATION 

The  DSVE  software  functions  much  like  ordinary 
CAD/CAE  software,  with  the  addition  that  it  works  in  a 
completely  immersive  virtual  environment.  For  the  system 
outlined  in  this  paper,  the  user  has  a  three-dimensional 
environment  rather  than  the  two-dimensional  environment  of  a 
typical  CAD/CAE  system.  In  this  three-dimensional 
environment,  human-computer  interaction  is  accomplished 
using  the  flightstick  for  most  user  actions.  Most  major 
functions,  such  as  navigating  the  scene  graph,  creating, 
modifying  and  deleting  geometry,  and  selecting  tools  are 
achieved  by  gesture  alone  using  the  flightstick. 

It  is  noteworthy  that  with  respect  to  other  multi-wall 
virtual  reality  applications,  DSVE  represents  a  departure  from 
current  conventions.  In  most  applications,  objects  (and  the 
virtual  environment  itself)  are  projected  so  that  the  user  views 
himself  or  herself  as  being  in  a  world  that  extends  beyond  the 
walls  of  the  display  device.  In  the  best  systems,  the  user  is 
induced  to  imagine  that  the  ‘landscape’  extends  infinitely  in 
every  direction.  In  contrast,  in  the  DSVE  synthesized 
geometry  is  suspended  in  mid-air,  approximately  one  meter  in 
front  of  the  GROTTO’s  front  wall.  This  creates  a  “design 
volume”  that  exists  roughly  in  the  middle  of  the  GROTTO. 
The  user  creates,  walks  around,  and  interacts  with  geometry 
displayed  within  this  volume.  Because  of  the  GROTTO’s 
layout,  the  front  wall  and  floor  are  the  primary  display 
surfaces.  However,  the  right  and  left  GROTTO  walls  are  also 
used  to  support  interaction. 

The  left  wall  of  the  GROTTO  contains  a  calculator  used  to 
assist  the  engineer  in  the  development  of  a  design.  The 
calculator  uses  the  XCALC  utility  commonly  available  in  the 
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X  Windows  environment.  Other  interaction  is  also  available 
on  the  left  wall  of  the  GROTTO  through  an  open  Irix  window, 
allowing  the  designer  communication  with  the  computer 
system.  Figure  2  shows  the  left  wall  of  the  GROTTO  with  the 
pulpit,  flightstick,  and  shutter  glasses. 

The  DSVE  implements  a  tool  handle/tip  metaphor  for 
interaction.  The  flightstick  represents  a  tool  handle,  into 
which  the  user  may  plug  tool  tips  to  accomplish  various  tasks. 
Interaction  is  facilitated  through  use  of  a  ray  for  selection  of 
tools  or  objects  in  the  immersive  environment.  Tracking 
algorithms  cause  the  ray  to  appear  to  the  user  to  emanate  from 
the  flightstick  and  to  point  in  the  proper  direction  at  all  times. 

On  the  GROTTO  right  wall  is  a  three-dimensional  tool 
cabinet  for  nine  tool  tips  used  for  interaction  with  the  design 
space.  Each  tool  tip  is  represented  by  a  three-dimensional  icon 
and  implements  one  form  of  interaction.  For  example,  the 
rotate  tool  tip  is  used  to  rotate  geometry  about  an  axis,  and  the 
sphere  tool  tip  is  used  for  generation  of  spherical  primitives. 
In  each  case,  appropriate  tool  tips  are  removed  from  the 
cabinet  for  use  with  the  tool  handle.  When  changing  tips, 
visual  feedback  is  provided  by  changing  the  graphical 
representation  of  the  tool  tip  in  the  virtual  environment. 
Pressing  a  button  (usually  the  Trigger)  on  the  tool  handle 
activates  the  tip  function.  Figure  3  shows  a  view  of  the  tool 
cabinet  from  inside  the  GROTTO,  and  Table  1  describes  tool 
tip  functions. 

The  current  version  of  the  DSVE  provides  three  major  tool 
tips  to  chose  from:  Select,  Create,  and  Modify.  The  Select  tool 
tip  is  represented  by  a  red  pointer  and  is  used  to  select  tools 
from  the  cabinet  and  geometry  from  the  virtual  workspace.  The 
Create  tool  tip  is  used  to  create  and  place  geometry  in  the 
design  environment.  When  the  Create  tip  is  active,  a  small 
representation  of  the  current  geometric  primitive  appears  at  the 
end  of  the  pointer.  The  Modify  tool  tip  is  represented  by  a 
hand  with  a  green  pointer  and  is  used  to  change  existing 
geometry  by  scaling,  rotating,  or  translating.  Once  the  Modify 
tip  is  active,  a  letter  appears  behind  the  hand  to  indicate  the 
function  (scale,  rotate,  translate)  that  is  current. 

Figure  4  shows  the  red  pointer  ray,  primarily  used  for 
selection  of  objects  from  the  tool  cabinet.  To  select  an  object, 
the  user  points  at  it  with  the  flightstick  and  pulls  the  Trigger. 
For  example  if  the  user  wishes  to  place  a  sphere  in  the 
workspace,  the  user  would  use  the  Selector  tip  to  point  at  the 
sphere  icon  in  the  cabinet.  Pressing  the  trigger  places  the 
Create  tip  on  the  tool  handle,  and  prepares  it  for  use  with  the 
sphere  geometric  primitive  as  shown  in  Figure  5. 

Changing  the  tool  tip  can  also  be  accomplished  by 
depressing  the  fire  button  on  the  flightstick.  Figure  6  shows 
the  result  of  selecting  the  translate  tool  tip.  Note  that  a  yellow 
capital  “T”  behind  the  hand  provides  visual  feedback  to  the 
change  in  function. 

In  the  current  version,  synthesis  of  geometry  is 
accomplished  using  geometric  primitives.  The  primitives 
currently  available  include  boxes,  cylinders,  cones,  spheres,  an 


annulus  referred  to  as  a  pipe,  and  NURBS  surfaces.  (The  pipe 
and  NURBS  surfaces  are  not  shown  in  this  paper).  A  primitive 
is  selected  for  use,  and  the  system  automatically  gives  the 
primitive  a  unique  name,  generates  the  primitive  with  unit 
dimensions,  and  places  the  primitive  at  the  origin  of  the  axis 
system.  The  primitive  can  then  be  manipulated  by  changing 
its  dimensions  through  scaling,  by  changing  its  orientation 
through  rotation,  and  by  translating  it.  Manipulations  are 
controlled  by  user  gestures  using  the  flightstick.  By  using  the 
scale,  rotate,  and  translate  functions  geometry  is  sized  and 
placed  in  desired  configurations  by  simple  gestures. 

Selection  of  primitives  for  use  with  the  Create  tip  is  shown 
in  Figure  7.  In  addition  to  selection  via  the  tool  cabinet,  the 
user  can  also  cycle  among  the  available  geometric  primitives 
by  pressing  the  left  flightstick  button.  When  the  desired 
primitive  is  shown,  the  user  depresses  the  flightstick  trigger  to 
place  a  copy  of  the  current  primitive  at  the  origin,  ready  for 
modification. 

The  Modify  tool  tip  is  used  to  change  existing  geometry  in 
the  design  volume.  Modifications  consist  of  scaling,  rotation, 
and  translation.  Tool  tip  selection  is  normally  accomplished 
by  interaction  with  the  tool  cabinet.  These  choices  may  be  also 
cycled  using  the  flightstick  left  button  in  a  manner  similar  to 
that  above.  Feedback  is  provided  in  the  form  of  a  large  letter 
corresponding  to  the  action:  T  is  for  translate,  R  is  for  rotate, 
and  S  for  scale.  Figures  6  and  8  illustrate  this  process. 

Synthesis  of  geometry  begins  by  selecting  a  geometric 
primitive  from  the  tool  cabinet.  With  the  desired  primitive 
shown  at  the  end  of  the  Create  tip,  the  user  presses  the 
flightstick  trigger.  The  system  then  automatically  gives  the 
primitive  a  unique  name,  generates  the  primitive  with  unit 
dimensions,  and  places  it  at  the  origin  of  the  axis  system. 
Once  located  at  the  origin,  primitives  can  then  be  manipulated 
by  changing  dimensions  through  scaling,  changing  orientation 
through  rotation,  and  by  translation  in  each  dimension. 
Figures  9  through  1 1  show  selecting  translate,  rotate,  and  scale 
functions,  respectively,  in  the  tool  cabinet. 

All  modifications  are  made  through  user  gestures.  At  the 
start  of  a  gesture,  the  flightstick  position  coordinates  and 
orientation  quaternion  are  read  from  the  magnetic  tracking  and 
stored  as  the  “start.”  As  the  flightstick  is  moved,  the  DSVE 
constantly  updates  the  gesture  vector  that  is  computed  based  on 
the  difference  in  position  and  orientation  between  the  current 
flightstick  position  and  that  at  the  start  of  the  gesture,  using 
tracker  data. 

When  scaling,  the  direction  of  the  gesture  indicates  the 
direction  of  the  scaling  effect,  and  the  magnitude  of  the  gesture 
is  used  to  determine  the  amount  of  scaling  applied.  To  scale  a 
component,  the  sequence  is:  a)  the  user  selects  the  Modify  tip 
and  chooses  the  scale  function;  b)  the  user  points  at  the 
component  and  pulls  the  trigger,  thereby  starting  the  gesture; 
c)  the  user  moves  the  flightstick  in  the  direction  and 
magnitude  of  the  desired  scaling  during  which  the  component 
is  re-scaled  in  real  time  to  match  the  user’s  input;  and  d)  upon 
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reaching  the  desired  result,  the  user  releases  the  trigger  thereby 
completing  the  scale  operation. 

When  rotating,  it  is  the  change  in  orientation  of  the 
flightstick  from  the  start  of  the  gesture  that  is  used  to  modify 
the  geometry.  To  rotate  a  component,  the  user:  a)  selects  the 
Modify  tip  and  chooses  the  rotate  function;  b)  the  user  points 
at  the  component  and  pulls  the  trigger,  thereby  starting  the 
gesture;  c)  the  user  twists  the  flightstick  in  the  direction  of  the 
desired  rotation  during  which  the  component  is  rotated  in  real 
time  to  match  the  user’s  input;  and  d)  upon  reaching  the 
desired  result,  the  user  releases  the  trigger  thereby  completing 
the  operation.  Rotation  was  originally  developed  to  provide 
rotation  about  one  of  the  axes  present  in  the  DSVE.  As  an 
alternate,  rotation  about  an  object’s  center  was  developed,  and 
research  is  continuing  with  regard  to  the  best  user  interface  for 
the  rotate  function. 

Translation  takes  place  in  a  similar  manner  to  scaling, 
except  that  the  gesture  moves  the  geometry  rather  than  scales 
it.  An  ‘Undo’  function  is  available  to  reverse  the  last 
modification. 

Figures  12  and  13  show  alteration  of  a  box  by  scaling 
along  each  of  two  axes,  while  Figure  14  shows  modifications 
to  a  cone.  In  each  case,  the  manipulations  are  controlled  by 
user  gestures  using  the  flightstick. 

As  action  to  create  and  modify  geometry,  primitives  are 
selected  and  placed  at  the  origin.  The  scale,  translate,  and 
rotate  functions  are  then  applied  to  geometric  primitives.  By 
using  the  scale,  rotate,  and  translate  functions  geometry  can  be 
sized  and  placed  in  desired  configurations  by  simple  gestures. 
Figures  15,  16,  and  17,  show  translation  of  a  box  and  a 
cylinder  by  gestures.  Further  details  can  be  found  in 
Maxwell’s  [9]  work. 

For  the  research  reported  in  this  paper,  the  flightstick  is 
used  as  the  principle  means  of  interaction.  However,  the  tool 
handle  and  tool  tip  metaphor  were  developed  purposefully  and 
specifically  to  allow  rapid  and  easy  conversion  to  other 
interface  devices  such  as  gloves,  haptic  feedback  devices,  and 
voice.  The  software  presently  has  provisions  for  future 
experimentation,  but  the  work  reported  upon  in  this  paper 
employs  the  flightstick  as  the  principal  means  of  interaction 
for  the  synthesis  of  geometry. 

Once  significant  three-dimensional  geometry  is 
synthesized,  it  can  be  saved  in  a  file.  For  this  action,  the 
keyboard  on  the  pulpit  is  used.  As  an  action  that  restores 
geometry  from  a  saved  file,  the  load  function  is  also  available 
at  the  keyboard  to  restore  three-dimensional  geometry  from  a 
file. 

Storage  of  geometric  data  currently  does  not  involve  solid 
modeling.  Although  the  research  team  has  recent  relevant 
experience  with  ACIS  solid  modeling  (see  [10]),  costly 
licensing  issues  cause  the  inclusion  of  solid  modeling  methods 
to  be  deferred  to  future  research.  Because  this  paper 
principally  focuses  on  the  use  of  an  immersive  environment  for 
geometric  synthesis,  provisions  were  made  for  future  use  of 


solid  modeling,  and  the  principle  research  thrust  remains 
investigation  of  the  synthesis  of  designs  in  a  virtual 
environment. 

In  addition  to  keyboard  action  for  saving  and  recalling 
geometry,  feature  information  related  to  the  geometry,  such  as 
tolerances,  can  be  entered  and  saved  for  each  unit  of  geometry. 
This  feature  based  information  storage  and  recall  provides  a 
means  of  data  management  that  would  be  useful  for  analysis, 
manufacturing,  and  other  product  realization  purposes. 


DEMONSTRATIONS 

To  illustrate  the  capabilities  of  the  DSVE,  experiments 
employing  the  DSVE  to  solve  mechanical  design  problems 
were  conducted.  Two  demonstrations  were  undertaken  to  gain 
experience  and  to  exercise  DSVE  features.  The  first 
demonstration  was  provided  by  an  experienced  naval  architect 
and  represented  an  actual  ship  geometric  design  problem.  The 
second  illustrative  problem  was  more  generic  and  provided 
additional  experiential  data  regarding  system  capabilities  and 
limitations. 

A  Ship  Design  Problem 

Equipment  located  on  a  specific  ship  class  has  experienced 
significant  problems  due  to  the  geometric  arrangement  of 
components  and  the  mechanical  relationships  between  power 
transmission  and  kinematic  machine  requirements.  As  a  first 
example,  the  problem  in  ship  design  was  undertaken  with  the 
DSVE,  and  the  DSVE  proved  helpful  in  the  construction  of  a 
geometric  model  and  analysis  of  the  mechanical  arrangement 
for  this  mechanical  system. 

The  Universal  Joint  Problem 

The  second  illustrative  problem  is  a  relatively  generic 
mechanical  design  example  based  on  a  common  universal 
joint.  The  geometric  synthesis  and  modeling  of  the  universal 
joint  is  a  well-known  mechanical  problem  to  which  many  can 
easily  relate.  For  this  reason,  work  with  the  universal  joint  is 
discussed  in  detail. 

As  an  illustration  of  this  capability,  the  geometry  of  a 
universal  joint  is  synthesized  starting  with  a  box.  At 
inception,  the  box  has  unit  dimensions  and  exists  at  the  origin. 
However,  it  is  reshaped  by  scaling  along  an  axis.  Also,  the  box 
is  repositioned  by  translation  as  shown  in  Figure  18.  The 
letter  “T”  is  displayed  to  signal  the  translation  operation. 

As  the  design  synthesis  process  continues,  geometric 
primitives  are  moved  or  shapes  are  altered  with  gestures.  This 
task  is  done  by  selection  using  the  red  pointer.  Upon  selection, 
the  primitive  is  moved  or  altered  using  gestures  as  needed 
using  the  scale,  translate,  and  rotate  capabilities.  As  the 
synthesis  process  continues,  additional  geometry  is  added.  In 
addition  to  the  box,  a  cylinder  is  generated,  and  the  cylinder  is 
scaled  and  translated  to  become  part  of  the  universal  joint, 
shown  in  Figure  19.  This  activity  occurs  through  gestures  with 
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the  flightstick.  Additional  primitives  are  added  as  the  design 
synthesis  process  continues,  as  seen  in  Figure  20.  As  an 
illustration  of  this  capability,  a  more  complete  universal  joint 
model  is  seen  in  Figure  21. 

As  a  three-dimensional  immersive  model,  examination 
and  review  of  this  geometry  from  any  angle  or  perspective  is 
possible.  The  user  can  even  view  an  object  from  inside  that 
object.  Because  the  viewpoint  from  the  pages  of  this  paper  is 
only  two-dimensional,  one  cannot  experience  this  effect,  but 
the  user  with  shutter  glasses  does  have  a  fully  immersive, 
three-dimensional  experience.  Review  of  the  synthesized 
geometry  in  the  DSVE  is  one  of  the  greatest  reasons  for  the  use 
of  an  immersive  design  paradigm.  Figures  22  through  24 
show  alternate  views  of  the  universal  joint,  with  each  view 
rendered  in  real-time  by  the  Onyx  2  computer  based  on  the 
user’s  position.  As  a  result,  the  designer  can  simply  move 
about  the  GROTTO  to  examine  the  product  from  any 
perspective. 

By  developing  geometry  for  a  mechanical  design  in  the 
DSVE,  it  is  believed  that  the  engineer  can  experience  and 
review  his  or  her  designs  in  new  ways.  This  spatial  feedback 
allows  more  rapid  and  effective  generation  of  a  higher  quality 
design,  and  holds  the  possibility  of  giving  new  tools  to 
engineers  that  design  and  construct  complex  mechanical 
systems. 


OBSERVATIONS  AND  CONCLUSIONS 

Experience  with  the  DSVE  provides  demonstration  of 
three  dimensional  design  synthesis  using  a  flightstick  for 
gestural  interaction.  The  DSVE  has  the  ability  to  create 
geometric  primitives  and  to  modify  them  to  create  more 
complex  pieces  of  geometry.  The  DSVE  is  fully  immersive 
utilizing  real-time  stereographic  rendering  and  magnetic 
tracking. 

Additionally,  the  pulpit  works  well  to  provide  keyboard 
input  while  inside  the  GROTTO.  However,  the  need  for 
keyboard  input  is  greatly  diminished  while  inside  the  DSVE, 
because  interaction  with  the  design  environment  occurs 
through  gestural  activity. 

Gesture  interaction  with  the  system  is  inexact.  This  is 
consistent  with  conclusions  reached  by  Vance,  Butler,  and 
Ebbesmeyer  [11].  Future  research  with  this  system  should 
provide  further  opportunities  for  experimentation.  Currently, 
inexactness  of  the  gesture  input  is  solved  by  keyboard  input. 
Gesture  input  is  not  meant  to  replace  the  keyboard  in  a 
CAD/CAE  system,  but  instead  to  provide  an  easy  to  use  3D 
interface  with  a  CAD/CAE  system. 

Tool  tip  navigation  is  cumbersome  without  the  addition  of 
visual  cues.  Visual  feedback  in  DSVE  is  essential  for  the 
typical  user.  Although  flightstick  button  equivalents  are 
available  for  most  of  the  icons  in  the  tool  cabinet,  the  cognitive 
load  associated  with  recognizing  which  buttons  to  push  on  the 
flightstick  and  at  what  time  to  push  them  is  too  much  for  some 


users.  Also,  significant  cognitive  load  is  placed  on  the  user  to 
maintain  geometric  consistency  within  the  DSVE. 

Viewing  complex  mechanical  models  inside  the  DSVE 
provides  a  new  and  unique  perspective  that  captivates  all  that 
view  it.  The  universal  joint  illustration  provided  excellent 
demonstration  material. 

The  development  of  the  DSVE  and  its  demonstration  on 
two  problems  in  mechanical  design  represents  experimentation 
with  a  new  technology  for  generation  of  designs  by  engineers. 
It  provides  an  initial  but  limited  experience  from  which  further 
research  can  be  conducted.  It  is  argued  that  the  use  of  a  three 
dimensional  interface,  such  as  the  system  provided  by  the 
DSVE  is  superior  to  the  user  interface  in  a  conventional  non- 
immersive  CAD/CAE  systems.  The  DSVE  system  permits  the 
designer  to  view  his  or  her  design  in  an  immersive 
environment  giving  a  much  richer  quality  of  visual  feedback 
during  the  design  synthesis  process.  The  gestural  interface 
provides  a  new  means  of  interaction  in  which  natural  human 
movement  in  the  immersive  environment  is  substituted  for 
multiple  keystrokes  and  2D  mouse  movement.  Although 
objective  statistical  data  is  not  yet  available  to  quantitatively 
substantiate  the  benefits  of  these  methods,  experience  with  the 
initial  demonstration  of  the  DSVE  system  leads  authors  to 
assert  that  the  DSVE  technology  holds  significant  promise. 

With  further  research  and  study,  the  human  computer 
interaction  issues  can  be  better  understood,  and  it  is  believed 
that  with  this  further  study  and  development,  the  DSVE 
approach  to  design  of  mechanical  systems  could  revolutionize 
the  manner  in  which  design  is  conducted.  Through  this 
revolution  in  technology,  engineers  are  offered  a  superior 
means  of  designing  ships,  aircraft,  and  other  complex 
mechanical  products.  The  result  is  better  quality,  improved 
performance,  and  a  substantial  increase  in  the  responsiveness 
of  the  engineering  design  function  to  the  needs  of  our  world. 
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Table  1  -  Tool  Tip  Names  and  Associated  Functions 


Tool  Tip 

Function 

Scale 

Scale  selected  geometry. 

Rotate 

Rotate  selected  geometry. 

Translate 

Translate  selected  geometry. 

Sphere 

Create  a  sphere  in  the  design  volume. 

Cone 

Create  a  cone  in  the  design  volume. 

Cube 

Create  a  cube  in  the  design  volume. 

Cylinder 

Create  a  cylinder  in  the  design  volume. 

Red  Pointer 

Enable  use  of  the  red  pointer. 

Figure  1  -  Dissection  of  the  Flight  Stick 


Figure  2  -  The  Pulpit  and  Left  Wall  of  the  GROTTO 


Figure  3  -  The  Cabinet  of  Tools 
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Figure  4  -  The  Red  Pointer 


Figure  5  -  Selection  of  the  Sphere  Primitive  from  the  Tool  Cabinet 
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Figure  6  -  Tool  Tip  Change  to  the  Translate  Function 


Figure  7  -  Selection  of  a  Cone 
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Figure  8  -  Changing  the  Tool  Tip  to  Scale  a  Sphere 


Figure  9  -  Modify  Tool  Tip  Set  for  Translating  Operations 
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Figure  10  -  Modify  Tool  Tip  Set  for  Rotating  Operations 


Figure  11  -  Modify  Tool  Tip  Set  for  Scaling  Operations 
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Figure  12  -  Scaling  of  a  Box  Along  the  First  Axis 


Figure  13  -  Scaling  of  a  Box  Along  the  Second  Axis 
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Figure  14  -  Scaling  Operations  on  a  Cone 


Figure  15  -  Translation  of  a  Box 


Figure  16  -  Translation  of  a  Cylinder 
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Figure  17  -  Further  Translation  of  a  Cylinder 


Figure  18  -  Repositioning  a  Box 


Figure  19  -  Adding  a  Cylinder  to  the  Design 
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Figure  20  -  Adding  Another  Box  to  the  Design 


Figure  21  -  A  Completed  Geometric  Model 


Figure  22  -  Viewing  the  Universal  Joint  from  a  Second  Perspective 
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Figure  23  -  Viewing  the  Universal  Joint  from  a  Third  Perspective 


Figure  24  -  Viewing  the  Universal  Joint  from  a  Fourth  Perspective 
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